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Abstract 

Efficiency of an optimization process is largely determined by the search algorithm and its 
fundamental characteristics. In a given optimization, a single type of algorithm is used in most 
Q_^ I applications. In this paper, we will investigate the Eagle Strategy recently developed for global 



optimization, which uses a two-stage strategy by combing two different algorithms to improve 
the overall search efficiency. We will discuss this strategy with differential evolution and then 
2 . evaluate their performance by solving real-world optimization problems such as pressure vessel 

and speed reducer design. Results suggest that we can reduce the computing effort by a factor 
of up to 10 in many applications. 
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kJ ■ 1 Introduction 

d , Metaheuristic optimization and computational modelling have become popular in engineering design 

and industrial applications. The essence of such paradigm is the efficient numerical methods and 
search algorithms. It is no exaggeration to say that how numerical algorithms perform will largely 
determine the performance and usefulness of modelling and optimization tools (Baeck et al.,1997; 
Yang, 2010). 

Among all optimization algorithms, metaheuristic algorithms are becoming powerful for solving 
tough nonlinear optimization problems (Kennedy and Eberhart, 1995; Price et al., 2005; Yang, 
2008; Cui and Cai, 2009). The aim of developing modern metaheuristic algorithms is to enable the 
capability of carrying out global search, and good examples of nature-inspired metaheuristics are 
particle swarm optimisation (PSO) (Kennedy and Eberhart, 1995) and Cuckoo Search (Yang and 
Deb, 2010a). Most metaheuristic algorithms have relatively high efficiency in terms of finding global 
optimality. 

The efficiency of metaheuristic algorithms can be attributed to the fact that they are designed 
to imitate the best features in nature, especially the selection of the fittest in biological systems 
which have evolved by natural selection over millions of years. In real- world applications, most data 
have noise or associated randomness to a certain degree, some modifications to these algorithms are 



Objective functions fi{x), ..., ffq{x) 

Initialization and random initial guess a;*^° 

while (stop criterion) 

Global exploration by randomization 

Evaluate the objectives and find a promising solution 

Intensive local search around a promising solution 

via an efficient local optimizer 

if (a better solution is found) 
Update the current best 

end 
Update t = t+l 
end 
Post-process the results and visualization. 



Figure 1: Pseudo code of the eagle strategy. 



often required, in combination with some form of averaging or reformulation of the problem. There 
exist some algorithms for stochastic optimization, and the Eagle Strategy (ES), develop by Yang 
and Deb, is one of such algorithms for dealing with stochastic optimization (Yang and Deb, 2010b). 
In this paper, we will investigate the Eagle Strategy further by hybridizing it with differential 
evolution (Storn, 1996; Storn and Price, 1997; Price et al., 2005). We first vahdate the ES by 
some multimodal test functions and then apply it to real-world optimization problems. Case studies 
include pressure vessel design and gearbox speed reducer design. We will discuss the results and 
point out directions for further research. 

2 Eagle Strategy 

Eagle strategy developed by Xin-She Yang and Suash Deb (Yang and Deb, 2010b) is a two-stage 
method for optimization. It uses a combination of crude global search and intensive local search 
employing different algorithms to suit different purposes. In essence, the strategy first explores the 
search space globally using a Levy flight random walk, if it finds a promising solution, then an 
intensive local search is employed using a more efficient local optimizer such as hill-climbing and 
downhill simplex method. Then, the two-stage process starts again with new global exploration 
followed by a local search in a new region. 

The advantage of such a combination is to use a balanced tradeoff between global search which 
is often slow and a fast local search. Some tradeoff and balance are important. Another advantage 
of this method is that we can use any algorithms we like at different stages of the search or even at 
different stages of iterations. This makes it easy to combine the advantages of various algorithms so 
as to produce better results. 

It is worth pointing that this is a methodology or strategy, not an algorithm. In fact, we can 
use different algorithms at different stages and at different time of the iterations. The algorithm 
used for the global exploration should have enough randomness so as to explore the search space 
diversely and effectively. This process is typically slow initially, and should speed up as the system 
converges (or no better solutions can be found after a certain number of iterations). On the other 
hand, the algorithm used for the intensive local exploitation should be an efficient local optimizer. 
The idea is to reach the local optimality as quickly as possible, with the minimal number of function 
evaluations. This stage should be fast and efficient. 



3 Differential Evolution 

Differential evolution (DE) was developed by R. Storn and K. Price by their nominal papers in 
1996 and 1997 (Storn, 1996; Storn and Price, 1997). It is a vector-based evolutionary algorithm, 
and can be considered as a further development to genetic algorithms. It is a stochastic search 
algorithm with self-organizing tendency and does not use the information of derivatives. Thus, it is 
a population-based, derivative-free method. Another advantage of differential evolution over genetic 
algorithms is that DE treats solutions as real-number strings, thus no encoding and decoding is 
needed. 

As in genetic algorithms, design parameters in a d-dimensional search space are represented as 
vectors, and various genetic operators are operated over their bits of strings. However, unlikely 
genetic algorithms, differential evolution carries out operations over each component (or each di- 
mension of the solution). Almost everything is done in terms of vectors. For example, in genetic 
algorithms, mutation is carried out at one site or multiple sites of a chromosome, while in differential 
evolution, a difference vector of two randomly-chosen population vectors is used to perturb an ex- 
isting vector. Such vectorized mutation can be viewed as a self-organizing search, directed towards 
an optimality. This kind of perturbation is carried out over each population vector, and thus can be 
expected to be more efficient. Similarly, crossover is also a vector-based component-wise exchange 
of chromosomes or vector segments. 

For a d-dimensional optimization problem with d parameters, a population of n solution vectors 
are initially generated, we have Xi where i = 1, 2, ..., n. For each solution Xi at any generation t, we 
use the conventional notation as 

^i — \^l,iJ^2,iT--T^d,i)^ (l) 

which consists of d-components in the d-dimensional space. This vector can be considered as the 
chromosomes or genomes. 

Differential evolution consists of three main steps: mutation, crossover and selection. 

Mutation is carried out by the mutation scheme. For each vector Xi at any time or generation 
t, we first randomly choose three distinct vectors Xp, Xg and Xr at t, and then generate a so-called 
donor vector by the mutation scheme 

vl+'=xl + Fixl-xl.), (2) 

where F e [0, 2] is a parameter, often referred to as the differential weight. This requires that the 
minimum number of population size is n > 4. In principle, F G [0, 2], but in practice, a scheme with 
F G [0, 1] is more efficient and stable. The perturbation d = F{xq — Xr) to the vector Xp is used to 
generate a donor vector Vi, and such perturbation is directed and self-organized. 

The crossover is controlled by a crossover probability Cr G [0, 1] and actual crossover can be 
carried out in two ways: binomial and exponential. The binomial scheme performs crossover on 
each of the d components or variables/parameters. By generating a uniformly distributed random 
number r^ G [0, 1], the jth component of Vi is manipulated as 

"*r = ^1.^ if ^^ < Cr (3) 

otherwise it remains unchanged. This way, each component can be decided randomly whether to 
exchange with donor vector or not. 

Selection is essentially the same as that used in genetic algorithms. It is to select the most fittest, 
and for minimization problem, the minimum objective value. 

Most studies have focused on the choice of F, Cr and n as well as the modification of (2). In 
fact, when generating mutation vectors, we can use many different ways of formulating (2), and this 
leads to various schemes with the naming convention: DE/x/y/z where x is the mutation scheme 
(rand or best), y is the number of difference vectors, and z is the crossover scheme (binomial or 
exponential). The basic DE/Rand/1/Bin scheme is given in (2). For a detailed review on different 
schemes, please refer to Price et al. (2005). 



4 ES with DE 

As ES is a two-stage strategy, we can use different algorithms at different stage. The large-scale 
coarse search stage can use randomization via Levy flights In the context of metaheuristics, the so- 
called Levy distribution is a distribution of the sum of N identically and independently distribution 
random variables (Gutowski, 2001; Mantegna, 1994; Pavlyukevich, 2007). 
This distribution is defined by a Fourier transform in the following form 

FN{k)^exp[~N\kf]. (4) 

The inverse to get the actual distribution L{s) is not straightforward, as the integral 

L{s) = - / cos(Ts)e"" ^"dr, (0 < /3 < 2), (5) 

I" Jo 

does not have analytical forms, except for a few special cases. Here L{s) is called the Levy distribu- 
tion with an index /3. For most applications, we can set a = 1 for simplicity. Two special cases are 
/? = 1 and (3 = 2. When /3 = 1, the above integral becomes the Cauchy distribution. When /3 = 2, it 
becomes the normal distribution. In this case. Levy flights become the standard Brownian motion. 
For the second stage, we can use differential evolution as the intensive local search. We know 
DE is a global search algorithm, it can easily be tuned to do efficient local search by limiting new 
solutions locally around the most promising region. Such a combination may produce better results 
than those by using pure DE only, as we will demonstrate this later. Obviously, the balance of local 
search (intensification) and global search (diversification) is very important, and so is the balance 
of the first stage and second stage in the ES. 

5 Validation 

Using our improved ES with DE, we can first validate it against some test functions which are highly 
nonlinear and multimodal. 

There are many test functions, here we have chosen the following 5 functions as a subset for our 
validation. 

Ackley's function 



f{x) = -20cxp 



d 

2 



V^.' 



1 "^ 
exp \- ^ cos(27rxi) + 20 + e, (6) 



where d = 1,2, ..., and —32.768 < Xi < 32.768 for i = 1,2, ...,d. This function has the global mini- 
mum /* = at a;, = (0, 0, ..., 0). 

The simplest of De Jong's functions is the so-called sphere function 

d 



f{x) = J2^l -5.12 <x,< 5.12, (7) 



whose global minimum is obviously /, = at (0,0, ...,0). This function is unimodal and convex. 
Rosenbrock's function 



d-l 



fix) = Y. [(^* - 1)' + 100(^^+1 - ^^f] ' (8) 



■[: 



whose global niinimuni /* = occurs at x^^ = (1,1,..., 1) in the domain —5 < Xi < b where 
I = 1, 2, ..., d. In the 2D case, it is often written as 

J{x,y)^{x-lf + im{v-x^)\ (9) 

which is often referred to as the banana function. 
Schwefel's function 

d 
f{x)^ -^XiSin(^J0\, -500 < x^ < 500, (10) 

whose global minimum /, w — 418.9829n occurs at Xi = 420.9687 where i = 1,2, ..., d. 
Shubert's function 

K 

f{x) = \^i cos (i + {i + l)x'] 

i=l 
K 

^icos(z+(z + l)j/)], (11) 

■ i=l 

which has multiple global minima /* « —186.7309 for K — b in the search domain —10 < x,y < 10. 
Table I summarizes the results of our simulations, where 9.7% corresponds to the ratio of the 
number of function evaluations in ES to the number of function evaluations in DE. That is the 
computational effort in ES is only about 9.7% of that using pure DE. As we can see that ES with 
DE is significantly better than pure DE. 

Table 1: Ratios of computational time 

Functions ES/DE 

Ackley {d ^ 8) 249% 

De Jong {d = 16) 9.7% 

Rosenbrock {d = 8) 20.2% 

Schwefel (d ^ 8) 15.5% 

Shubert 19.7% 



6 Design Benchmarks 

Now we then use the ES with DE to solve some real-world case studies including pressure vessel and 
speed reducer problems. 

6.1 Pressure Vessel Design 

Pressure vessels are literally everywhere such as champagne bottles and gas tanks. For a given 
volume and working pressure, the basic aim of designing a cylindrical vessel is to minimize the total 
cost. Typically, the design variables are the thickness di of the head, the thickness d2 of the body, 
the inner radius r, and the length L of the cylindrical section (Coello, 2000; Cagnina et al., 2008). 
This is a well-known test problem for optimization and it can be written as 

minimize f(x) = Q.6224dirL + 1.7781d2r2 

+3.1661dlL + 19Mdlr, (12) 

subject to the following constraints 



gi{x) = -rfi+0.0193r< 

g2ix) = -rf2 + 0.00954r < 

gsix) = -rrr^L - ^r^ + 1296000 < 

gilx) = L - 240 < 0. 



4iL^3 1 1 oQcnnn ^ n ^ '^i 



The simple bounds are 

0.0625 <di,d2<99x 0.0625, (14) 

and 

10.0 < r, L< 200.0. (15) 



Table 2: Comparison of number of function evaluations 
Case study Pure DE ES ES/DE 

Pressure vessel 15000 2625 17.7% 

Speed reducer 22500 3352 14.9% 

Recently, Cagnina et al. (2008) used an efficient particle swarm optimiser to solve this problem 
and they found the best solution /* « 6059.714 at 

x^ « (0.8125, 0.4375, 42.0984, 176.6366). (16) 

This means the lowest price is about $6059.71. 

Using ES, we obtained the same results, but we used significantly fewer function evaluations, 
comparing using pure DE and other methods. This again suggests ES is very efficient. 

6.2 Speed Reducer Design 

Another important benchmark is the design of a speed reducer which is commonly used in many 
mechanisms such as a gearbox (Golinski, 1973). This problem involves the optimization of 7 vari- 
ables, including the face width, the number of teeth, the diameter of the shaft and others. All 
variables are continuous within some limits, except 2:3 which only takes integer values. 

f{x) = 0.7854a;ia;^(3.3333a;^ + 14.9334x3 - 43.0934) 

-1.508a;i(a;^ + x^) + 7.4777(a;^ + x^) 

(17) 

(18) 
(19) 

(20) 
(21) 
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397.5 
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-1 <o 


gsix) == 


1.93a;| 

X2X3xl 


-i<o. 


giix) = 


1.93a;| 

X2XsX'^ 


-1 <0 



1.0 // 745.0x4 
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g^{x) ^ 7777^^/ ( J^ ] + 16.9 x lO^ - 1 < (22) 



, , 1.0 //745.0x5\2 ^ ^ ^ 

^«(^)-85^V(^^) +15^-5x10^-1^0 (23) 

97{x) = ^ ~ 1 < (24) 

g^{x) = ^ - 1 < (25) 

Xi 



5.(-)-^-l<0 (26) 

gM='-^^^^^±^-l<0 (27) 

, , 1.1x7 + 1.9 , , 

gii(a;) = ^ 1<0 28 

2:5 

where the simple bounds are 2.6 < x\ < 3.6, 0.7 < X2 < 0.8, 17 < xs < 28, 7.3 < X4 < 8.3, 
7.8 < xs < 8.4, 2.9 < X6 < 3.9, and 5.0 <x-j < 5.5. 

In one of latest studies, Cagnina et al. (2008) obtained the following solution 

x^ = (3.5, 0.7, 17, 7.3, 7.8, 3.350214, 5.286683) (29) 

with /„ii„ = 2996.348165. 

Using our ES, we have obtained the new best 

a;, = (3.5, 0.7, 17, 7.3, 7.8, 3.34336449, 5.285351) (30) 

with the best objective /^in = 2993.7495888. We can see that ES not only provides better solutions 
but also finds solutions more efficiently using fewer function evaluations. 

7 Discussions 

Metaheuristic algorithms such as differential evolution and eagle strategy are very efficient. We 
have shown that a proper combination of these two can produce even better performance for solving 
nonlinear global optimization problems. First, we have validated the ES with DE and compared 
their performance. We then used them to solve real-world optimization problems including pressure 
vessel and speed reducer design. Same or better results have been obtained, but with significantly 
less computational effort. 

Further studies can focus on the sensitivity studies of the parameters used in ES and DE so as to 
identify optimal parameter ranges for most applications. Combinations of ES with other algorithms 
may also prove fruitful. Furthermore, convergence analysis can provide even more profound insight 
into the working of these algorithms. 
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